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ABSTRACT 

The  Combined  Arms  Tactical  Training  Simulator  (CATTS) 
is  studied  with  a view  towards  evaluating  the  training  ef- 
fectiveness of  command  and  control  instruction.  The  objec- 
tives and  goals  of  the  CATTS  system  are  reviewed,  as  well 
as  the  training  system  itself.  Concepts  and  methodology 
for  the  evaluation  of  the  CATTS  system  are  presented,  with 
a proposed  test  procedure  outlined. 
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I . INTRODUCTION 


Technological  advances  since  the  last  world  war  have 
helped  to  radically  change  the  scope,  pace  and  quality  of 
life  today.  Now  it  is  possible  to  have  round-the-world 
conversations  in  a matter  of  seconds,  and  it  is  also  possible 
to  circumnavigate  the  globe  in  a matter  of  minutes.  Such 
changes  have  totally  revamped  the  way  people  and  nations 
think  and  act.  Significant  changes  have  occurred  in  concepts 
about  travel,  communications,  business  and,  even  in  one  of 
man's  oldest  activities,  warfare. 

Today's  technological  improvements,  along  with  the  pro- 
mise of  tomorrow's  advances,  have  forced  senior  military 
commanders  to  realize  that  there  will  be  no  periods  of  train- 
ing available  before  deployment  and  few  opportunities  to 
learn  from  the  mistakes  of  allies  as  were  experienced  during 
the  past  two  world  wars.  Indeed,  the  victor  of  the  next 
major  confrontation  will  most  likely  be  the  one  with  the 
quickest  reaction  time  and  the  best  preparedness  in  resources 
and  training.  This  is  especially  evident  in  light  of  the  im- 
portance placed  on  the  tactics  of  surprise  attack  by  the 
Soviet  and  Warsaw  Tact  countries.  More  recently,  the  experi- 
ences of  the  last  two  Mid-East  wars  have  highlighted  the 
crucial  importance  of  military  preparedness  in  the  modern- 
day  conflict. 
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The  outcome  of  all  major  ancient  and  modern  military 
conflicts  when  analyzed  in  any  depth  are  seen  to  be  based 
on  preparedness — not  luck  [20]  . As  the  modern  army  becomes 
more  technologically  sophisticated  and  as  the  potential 
battlefield  becomes  more  diverse,  the  problem  of  training 
becomes  more  complex  and  more  critically  important  to  solve. 
The  commanders  who  will  manage  this  complex  battlefield  must 
be  very  well  trained  in  controlling  the  myriad  of  advanced 
weapons  systems  and  equipment  and  also  of  analyzing  the 
copious  information  available  to  them. 

Traditionally,  the  only  way  to  become  trained  in  battle- 
field command  and  control  was  either  by  actual  combat  experi- 
ence or  by  field  maneuvers.  The  former  is  rather  infrequent 
and  the  latter  is  very  expensive  and  falls  short  of  portray- 
ing the  broad  spectrum  of  combat  conditions  which  may  exist 
worldwide.  The  inexperienced  brigade  or  battalion  commander 
sent  to  distant  and  unknown  lands  faces  formidable  problems 
for  which  field  maneuvers  can  only  partially  prepare  him  at 
best.  This  situation  has  created  the  need  for  a training 
system  which  can  augment  the  traditional  field  exercises 
and  classroom  learning  by  offering  a training  experience 
based  on  realism  with  regard  to  the  critical  elements  of 
terrain,  resources,  weather,  personnel,  and  enemy  tactics. 
Just  such  a training  system  is  the  Army's  Combined  Arms 
Tactical  Training  Simulator  (CATTS) . 

As  was  mentioned  previously,  one  way  to  become  trained 
in  battlefield  command  was  through  actual  combat  experience. 
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Yet  a study  by  the  Pentagon's  Systems  Analysis  Office  [20] 
during  the  last  years  of  the  Vietnam  conflict  revealed  that 
casualty  rates  in  U.S.  battalions  commanded  by  new  and  in- 
experienced officers  was  twenty  per  cent  higher  than  in  those 
battalions  commanded  by  more  experienced  officers.  Obviously, 
that  is  an  unacceptable  rate.  This  statistic  pointed  out  to 
senior  military  commanders  that  the  training  received  by 
young  officers  was  not  sufficient  to  help  overcome  their 
lack  of  experience. 

Until  a few  years  ago  the  training  received  by  prospec- 
tive battalion  and  brigade  commanders  was  limited  to  class- 
room exercises  on  the  theory  of  command  and  control,  along 
with  the  standard  courses  of  military  tactics  and  strategy 
and  the  principles  of  leadership  [20] . The  only  opportunity 
for  a battalion  commander  to  put  the  theory  in  practice  with- 
in a training  environment  was  during  a command  post  exercise 
(CPX)  or  a field  training  exercise  (FTX) . This  was  only 
possible  after  the  officer  assumed  command  of  a functioning 
unit.  A command  post  exercise  is  a manual  simulation,  that 
is,  the  computations  involved  must  be  done  by  hand,  and  al- 
though some  are  aided  by  high  capacity  calculators  (mini- 
processors) , they  all  suffer  to  one  degree  or  another  from 
the  main  drawback  of  the  CPS:  lack  of  realism  [10].  A 
field  training  exercise  (FTX)  is  a training  device  used  to 
acquaint  the  commander  and  his  staff  with  the  principles, 
procedures,  and  problems  of  tactical  field  maneuvers  and 
combat  techniques  while  actually  controlling  his  unit; 
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fully  functioning  and  fully  manned  and  equipped  [7] . How- 
ever, in  peacetime  field  training  exercises  normal  peace- 
time priorities  are  in  effect  and  the  administrative  and 
logistical  details  of  garrison  life  must  still  be  observed. 
This  is  a distraction  that  can  only  be  eliminated  during 
combat.  Furthermore,  in  a field  training  exercise  the  com- 
mander cannot  get  involved  in  the  fast-moving  situations 
characteristic  of  warfare  and  also,  the  weapons  systems  can- 
not fire  in  a way  that  the  true  effect  can  be  measured. 

Weapons  effects  must  be  simulated,  since  it  is  culturally 
unacceptable  to  have  two-sided  live-fire  exercises.  Thus, 
weapons  effects  (and  their  attendant  operational  effects) 
are  not  part  of  field  exercises. 

The  key  to  the  effectiveness  of  CATTS  as  a training  system 
is  that  the  realism  that  was  missing  from  the  aforementioned 
traditional  methods  has  now  been  achieved.  Using  this  train- 
ing system  the  commander  and  his  staff  must  evaluate  in- 
formation from  the  subordinate  units  and  make  decisions  at 
the  same,  often  man-killing,  tempo  as  they  would  in  actual 
combat.  It  is  the  purpose  of  this  thesis  to  describe  the 
CATTS  system  and  to  propose  methodology  and  concepts  for  the 
evaluation  of  such  a system. 
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II.  OBJECTOVES  OF  THE  COMBINED  ARMS  TACTICAL 


TRAINING  SIMULATOR 

A.  PROBLEMS  WITH  TRADITIONAL  TRAINING  TECHNIQUES 

Historically,  the  best  battlefield  commanders  have  been 
those  that  have  weathered  the  tribulations  of  combat  pre- 
viously, those  commanders  that  have  faced  and  overcome  the 
stresses  of  command  under  pressure  from  actual  conflict  [20] . 
It  is,  of  course,  impossible  to  have  seasoned  veterans  in  all 
key  command  positions  and  it  is  just  as  difficult  to  train  a 
peacetime  commander  in  the  skills  and  techniques  of  command 
which  can  be  acquired  only  through  actual  combat.  It  is  for 
these  reasons  that  a more  realistic  training  approach  was 
desperately  needed  by  the  Army;  an  approach  that  succeeded 
where  the  command  post  exercise  and  the  field  training  exer- 
cise failed. 

There  are  a number  of  teaching  and  training  methods  that 
have  been  used  in  the  past  [7],  Some  of  the  most  common  are; 


1. 

Terrain  Model 

Exercise 

2. 

Map  Exercise 

3. 

Terrain  Exercise 

4. 

Tactical  Drill 

Exercise 

5. 

Map  Maneuver 

6. 

Command  Post  Exercise 

7. 

Field  Exercise 

8. 

Field  Maneuver 
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Tactical  Exercise  Without  Troops 
10.  Unconventional  Warfare  Training  Exercise 

All  of  these  map  and  tactical  training  exercises  vary 
greatly  in  content  and  in  scope.  These  exercises  may  be  one- 
sided or  two-sided.  In  a one-sided  exercise  the  opposing 
force  is  represented  by  an  umpire  or  a controller  who  makes 
decisions  on  casualty  assessments,  maneuver  and  tactical  ef- 
fectiveness, etc..  In  a two-sided  exercise  the  forces  involved 
maneuver  against  each  other  without  firing  live  ammunition. 

In  this  case  the  controllers  or  umpires  monitor,  judge  and 
arbitrate  engagements  of  both  sides  to  keep  the  exercise  with- 
in the  stated  objectives.  Any  of  these  tactical  exercises 
listed  above  may  be  of  two  modes:  free  or  controlled.  In  a 
free  exercise  both  sides  are  permitted  total  freedom  of  opera- 
tion during  the  course  of  the  exercise.  However,  in  a con- 
trolled exercise  one  or  both  sides  are  constrained  to  act 
according  to  a preconceived  scenario  which  insures  that  the 
objectives  of  the  exercise  are  fulfilled.  As  was  previously 
noted,  the  umpires  or  controllers  of  these  tactical  exercises 
are  responsible  for  the  evaluation  of  the  quality  of  perfor- 
mance of  individuals,  units,  staffs,  equipment  and  weapons 
systems;  and  the  adequancy  of  concepts,  procedures,  and  tech- 
niques employed  in  the  exercise. 

Effectively  controlling  a tactical  exercise  while  simul- 
taneously permitting  the  tactical  exercise  scenario  to  real- 
istically portray  the  actual  combat  environment  presents  a 


most  difficult  task  for  exercise  control  personnel.  More 
significantly,  as  the  complexity  of  the  controllers'  job 
increases  and  more  demands  are  placed  upon  them,  the  realism 
of  the  entire  exercise  is  placed  in  jeopardy.  The  value  of 
the  exercise  as  a training  medium  is  often  directly  propor- 
tional to  the  degree  of  realism  portrayed  in  the  tactical 
exercise.  This  then  is  the  single  most  important  deficiency 
of  the  conventional  tactical  exercises — the  inability  to  de- 
pict the  actual  battlefield  situation  with  a high  degree  of 
fidelity.  As  S.  L.  A.  Marshall  once  wrote:  "It  is  not  with- 
in the  ingenuity  of  man  ever  to  fully  close  the  gap  between 
training  and  combat." [12] 

It  is  also  a very  well  known  fact  that  many  untried  com- 
manders who  were  once  classified  as  competent  theoretical  tac- 
ticians have  met  defeat  when  they  encountered  the  realities 
of  armed  conflict.  What,  therefore,  separates  the  successful 
commanders  from  those  who  failed  to  meet  and  overcome  the 
challenge?  Colonel  Wesley  W.  Yale  has  written: 

"The  art  of  tactical  control  is,  therefore,  more 
than  a study.  It  rests  on  practice.  Just  as  an 
artist  cannot  paint  a picture  solely  by  reading 
books  about  Michelangelo,  a commander  cannot  train  him- 
self by  reading  about  the  great  captains. 

"The  control  of  battle  is  more  a matter  of  tech- 
niques than  of  tactics — the  positioning  of  the  com- 
mander at  various  stages  of  action,  the  organization 
and  use  of  a mobile  staff,  the  measures  needed  to 
ensure  precise  timing  of  fire  support  with  maneuver 
and  the  correct  and  timely  use  of  ground  and  air  com- 
mand transport.  In  addition,  perhaps  obviously,  there 
must  be  physical  familiarity  with  the  tools  of  control 
such  as  radio  and  computer  input/output  display.  Less 
obviously,  there  is  a vital  need  to  understand  the  or- 
ganization of  communications  and  communications 
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personnel,  to  the  end  that  the  commander  is  not 
snowed  under  by  trivia,  but,  rather,  works  only  with 
essential,  decisionmaking  information."  [20] 

The  United  States  Army  has  recognized  the  lack  of  realism 
in  those  map  and  tactical  exercises  mentioned  previously* 
and,  furthermore,  it  has  also  recognized  that  these  "tech- 
niques" that  Colonel  Yale  mentioned  are  the  key  to  fully 
preparing  today's  commanders  for  the  battlefield  of  tomorrow. 

It  was  toward  this  end  that  the  Combined  Arms  Tactical  Train- 
ing Simulator  (CATTS)  was  designed. 

The  CATTS  system  was  developed  for  the  Army  by  TRW  Sys- 
tems Group,  Redondo  Beach,  California.  This  system  is  based 
on  the  concept  that  it  is  within  the  state  of  the  art  to  model 
battlefield  conditions  and  conflict  outcomes  to  an  accuracy 
level  which  is  commensurate  with  the  training  needs  of  bat- 
talion commanders  and  their  staffs.  In  effect,  the  CATTS 
system  provides  realistic  assessments  for  the  consequences 
of  decisions  by  the  trainee  battalion  commander  by  using 
complex  and  detailed  computer  routines  which  simulate  not 
only  the  basic  tactical  unit  functions,  but  also  the  inter- 
play of  these  units  with  the  selected  battlefield  scenario, 
enemy  situation,  weather  conditions,  etc. 

B.  SIMULATION  TRAINING 

History  indicates  that  one  of  the  most  decisive  factors 
in  winning  and  losing  wars  is  the  adequacy  of  training  and 
the  motivation  of  soldiers,  sailors,  airmen  and  marines  who 
make  up  the  combat  forces.  However,  it  has  been  very  difficult 
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to  achieve  and  maintain  a constant  high  state  of  combat 


readiness  in  peacetime  without  jeopardizing  the  safety  of  the 
trainees  and  their  equipment  and  incurring  prohibitive  train- 
ing costs.  Further,  the  rather  high  turnover  rate  of  person- 
nel has  been  a significant  impediment  to  achieve  this  high 
state  of  preparedness.  High  morale  in  combat  units  during 
peacetime  is  directly  proportional  to  the  state  of  training 
in  these  units:  the  better  prepared  a unit  is  for  a certain 
mission,  the  more  willing  it  will  be  to  perform  such  a mission. 
New  advances  in  technology  have  provided  revolutionary  capabi- 
lities for  safer,  more  economical,  and  effect We  training  in 
operational,  maintenance,  and  combat  skills  by  the  use  of 
simulation. 

Most  simulators  today  in  the  Department  of  Defense  are 
used  to  help  teach  people  how  to  operate  a major  weapon  sys- 
tem, i.e.,  a piece  of  hardware — how  to  fly  an  airplane,  drive 
a tank,  or  dock  a ship.  In  the  civilian  sector,  simulators 
are  used  in  high-school  driver  education  classes  to  teach 
students  how  to  drive  cars.  They  are  also  used  by  the  air- 
lines for  pilot  proficiency  training. 

The  Defense  Department  is  currently  developing  or  has 
developed  a whole  family  of  simulators  which  can  provide 
realistic,  effective  training  at  reduced  costs.  At  one  end 
of  the  simulation  spectrum  there  are  the  pilot  and  aircrew 
simulators  which  provide  a high  degree  of  realism  in  visual 
and  motion  cues.  These  devices,  with  their  impressive  array 
of  mechanical,  electronic,  and  visual  subsystems,  have 
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received  the  most  public  attention.  The  Advanced  Simulator 
for  Undergraduate  Pilot  Training  (ASUPT)  developed  by  the 
Air  Force  Human  Resources  Laboratory  [16]  is  an  example  of 
this  modern  technology.  This  simulator  embodies  the  state 
of  the  art  in  technology  at  this  point  and  is  the  epitome  of 
today's  flight  simulation.  As  the  trainee  sits  in  the  simu- 
lator cockpit,  he  sees  a wrap-around  computer-generated  image 
which  is  spatially  accurate  with  regard  to  the  airborne  flight 
and  ground  track  of  the  simulated  aircraft.  The  image  also 
is  coupled  to  the  motion  base  of  the  simulator  so  that  the 
visual  scene  accurately  depicts  positional  information  of  the 
real  world. 

At  the  other  end  of  the  spectrum,  clever  applications 
of  modern  technologies — eye-safe  lasers,  cheap  microproces- 
sors, and  new  advances  in  low-cost  electronic  devices — have 
made  possible  realism  in  combat  engagement  simulation  even 
for  infantry  and  armor.  The  probability  of  casualties  among 
combat  troops  is  greatest  during  the  first  few  weeks  of  com- 
bat experience.  In  close  combat  concealment  and  cover  are 
the  keys  to  survival.  Exposure  to  the  lethality  of  modern 
weapons  with  their  high  accuracy  and  firepower  means  high 
casualty  rates.  Therefore,  new  training  techniques  are  need- 
ed to  simulate  combat  conditions  realistically  between  two 
opposing  forces  and  to  teach  soldiers  to  do  the  correct 
things  instinctively  in  the  event  of  an  actual  armed  con- 
flict. 
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In  order  to  provide  this  realistic  combat  training,  the 
Department  of  the  Army  has  developed  a system  based  on  low- 
power  lasers  and  inexpensive  microprocessors  to  teach  battle 
skills  for  a two-sided  combat  situation.  This  system  is 
called  the  Multiple  Integrated  Laser  Engagement  System 
(MILES) . This  system  actually  simulates  the  exposure  of  a 
soldier  to  the  lethality  of  weapons,  provides  data  for  a cri- 
tique of  a two-sided  engagement,  and  permits  training  re- 
petition. 

Training  units  are  furnished  with  rifles,  machine  guns, 
tank  and  antitank  guns  that  are  equipped  with  eye-safe  lasers. 
Sensors  are  mounted  on  each  soldier,  vehicle,  and  weapon  and 
connected  to  a microcomputer  carried  by  each  man  or  weapon. 

On  the  infantryman  the  sensors  and  the  microcomputer  are 
mounted  on  the  belt  harness  and  helmet.  When  a weapon  is 
actuated,  a blank  round  is  fired  by  the  weapon  and  a light 
beam  containing  a distinctive  code  is  emitted  from  the  laser. 
Any  sensor  intercepting  the  beam  records  a lethal  hit  if  the 
sensor  is  located  in  an  area  where  a hit  from  that  kind  of 
weapon  would  normally  disable  the  target.  The  soldier's 
computer  is  programmed  so  that  a hit  in  a vital  area  from  any 
weapon  is  likely  to  be  a kill.  The  microcomputer  informs 
the  bearer  if  he  has  been  hit  and  automatically  disables  his 
weapon,  removing  him  from  the  exercise. 

The  results  of  preliminary  exercises  with  this  type  of 
system  have  been  most  impressive  [10]  . It  has  generated 
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real  enthusiasm  from  recruits  and  experienced  veterans  aL ike. 
This  system  puts  new  challenges  and  excitement  into  military 
field  training  and  teaches  skills  that  formerly  could  only  be 
learned  in  battle. 

The  advantages  of  simulation  training  are  not  restricted 
to  training  effectiveness  alone.  An  equally  important  aspect 
of  simulation  training  is  the  greatly  reduced  cost  with  an 
associated  high  degree  of  training  effectiveness.  In  the  par- 
lance of  Operations  Research,  such  systems  have  proven  to  be 
cost  effective.  The  Department  of  Defense  has  accelerated 
the  procurement  of  flight  simulators  and  increased  their  use 
in  both  training  and  operational  squadrons  [16] . The  original 
motivation  for  this  step  can  be  found  in  the  fact  that  fuel 
costs  have  increased  well  over  300  per  cent  since  the  last 
Mideast  War  of  1973.  The  record  shows  that  in  fiscal  1974 
the  Air  Force  flew  about  one  million  hours  less  than  in  fis- 
cal 1973,  but  it  cost  one  billion  dollars  more  to  fly  these 
reduced  hours  [16] . The  Defense  Department  has  consequently 
submitted  to  Congress  a procurement  request  of  almost  one 
billion  dollars  for  the  fiscal  1977,  1978,  and  1979  years  for 

simulators  for  new  and  existing  aircraft  [16] . 

' 

Studies  on  the  comparison  of  the  operating  costs  of 
seventeen  aircraft  and  their  respective  simulators  indicates 
that,  at  one  extreme,  aircraft  operating  costs  are  thirty 
times  greater  than  simulator  operating  costs,  while  at  the 
other  end  of  the  spectrum,  the  operating  costs  for  certain 


helicopters  are  only  three  times  greater  [16] . The  median 


value  shows  that  aircraft  operating  costs  are  over  ten  times 
greater.  The  Department  of  Defense  anticipates  that  the  over- 
all fiscal  1978  flying-hour  savings  due  to  the  use  of  all 
flight  training  simulators  to  be  676,000  hours  which  will 
also  result  in  the  savings  of  566  million  gallons  of  fuel. 

Simulation  training  seems  to  be  both  training  and  cost 
effective  in  the  particular  instance  of  flight  systems  simu- 
lations;  however,  the  payoff  for  the  military  may  be  just  as 
great  in  the  field  of  combat  engagement  simulations  as  indi- 
cated in  Commanders  Digest  of  15  August  1974: 

"Simulation  equipment  used  in  both  ground  and 
aviation  related  training  has  unlimited  potential  and 
provides  an  innovative  and  cost-effective  teaching 
tool.  This  equipment  employs  the  latest  state-of- 
the-art  in  educational  technology  and  methodology. 

"Although  ground-related  simulation  equipment 
has  not  yet  been  developed  to  the  high  state  of 
sophistication  of  aviation-related  simulation  equipment, 
ground-related  simulators  are  designed  to  improve  the 
learning  process  and  to  assist  in  producing  a better 
trained  individual.  They  are  neither  designed  nor 
intended  to  replace  hands-on  training,  but  rather  to 
make  the  hands-on  training  more  meaningful  and  effective." 

It  is  just  this  type  of  ground-related  simulation  which 
the  Army  is  employing  in  the  CATTS  system.  The  training 
is  aimed  at  the  brigade  and  battalion  level  and  seeks  to 
implant  an  appreciation  of  tactical  requirements  during  a 
moving  situation.  In  the  recent  past,  the  Amy,  Navy,  and 
the  Marine  Corps  have  developed  [10]  some  highly  sophisti- 
cated electronic  simulation  devices  for  command  training; 
however,  these  tools  do  not  get  to  the  root  of  the  command 
and  control  problem.  To  repeat,  it  is  not  so  much  tactical 
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training  that  is  needed  as  is  the  methods  or  techniques  of 
exercising  command.  The  requirements  for  these  techniques 
are  greatest  at  the  brigade  and  battalion  task  force  levels. 
Here  the  prospective  leader  is  a field  grade  officer  schooled 
in  the  theory  of  military  tactics  without,  in  all  probability 
mastering  the  techniques  of  command.  It  is  this  officer  who 
is  the  prime  target  for  simulation  leadership  training. 
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III.  CATTS  SYSTEM  OVERVIEW 


A.  THE  SYSTEM 

The  present  day  commander  has  many  more  variables  to  deal 
with  and  generally  much  less  time  to  deal  with  them  than  his 
counterpart  of  just  a few  decades  ago.  Such  variables  as  the 
weather  and  the  terrain  play  as  important  a role  as  always, 

I 

but  today  the  commander  must  also  be  knowledgeable  about  the 
latest  weapons  systems,  sensors,  rockets,  missiles,  satellites, 

' 

lasers,  etc.  The  Combined  Arms  Tactical  Training  Simulator 
provides  the  means  to  realistically  portray  all  of  these 
variables  and  the  terrain  and  the  weather.  It  was  conceived 
as  a solution  to  the  problem  of  providing  effective  training 
for  battalion  field  commanders  and  their  staff  officers.  In 
particular,  the  desire  was  for  an  automated  training  aid  which 
would  approximate  the  decision-making  experience  which  can 
now  be  obtained  only  through  actual  participation  in  combat 
operations. 

The  battalion  commander  and  his  staff  (also  referred  to 
as  players  or  as  trainees)  are  housed  in  a realistic  mock-up 
of  the  battalion  Tactical  Operations  Center  (TOC) , which  con- 
tains fully  functional  government-provided  radios  and  field 
telephones  modified  and  connected  to  a sophisticated  solid 
state  communications  system.  The  TOC  also  contains  a simu- 
lated radio  teletype  (RATT) , microphone  monitor  pickups,  a 
public  address  system,  and  a multi-directional  sound  system. 
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All  of  these  devices  are  monitored  and  controlled  by  a team 
of  controllers. 

The  trainees  give  commands  to  and  receive  reports  from 
these  controllers  who  also  act  as  subordinate  commanders 
(company,  platoon) . However,  instead  of  these  "subordinate 
commanders"  dealing  with  real  troops,  real  enemies  and  a 
real  environment,  they  perform  the  equivalent  function  but 
deal  with  the  computer  instead.  Figure  1 shows  this  equiva- 
lence in  a diagrammatic  fashion.  When  a trainee  uses  one  of 
the  communications  devices,  he  communicates  with  a controller 
who  plays  the  role  of  the  person  who  would  normally  be  at 
that  particular  communications  net  (i.e.,  higher,  lower, 
adjacent  commands,  artillery,  air  support,  etc.) . Conversa- 
tions anywhere  within  the  TOC  can  be  monitored  by  any  con- 
troller. All  voice  communications  are  recorded  on  a twenty- 
track  audio  recorder.  Each  radio  network  may  have  static 
and/or  jamming  added  to  it  at  the  discretion  of  the  controllers. 
Directional  battle  and  motor  noises  can  be  introduced  by  the 
controllers  over  the  multi-directional  speaker  system. 

The  training  environment  provided  is  a physically  real- 
istic one.  The  trainees  operate  in  a physically  familiar 
setting  using  familiar  communications  equipment  to  communi- 
cate with  the  outside  world.  Their  only  source  of  informa- 
tion about  the  course  of  the  battle  comes  over  the  communica- 
tions system.  From  information  provided  by  the  controllers, 
the  trainees  must  work  as  a team  to  maintain  '.n  accurate 


j 


22 


I 

1 


picture  of  what  is  going  on,  make  command  decisions  under 
stress  in  real  time,  and  communicate  their  decisions  to  the 
controllers  playing  the  appropriate  roles. 

The  technical  problem  the  CATTS  system  attempts  to  solve 
is  that  of  giving  the  controllers  an  aid  to  calculate  battle 
outcomes  rapidly  enough  and  realistically  enough  for  training 
needs  without  constraining  the  freedom  of  action  of  the  train- 
ees. This  is  accomplished  through  the  use  of  a large-scale 
computer  system  on  which  the  battle  is  simulated  by  a mathe- 
matical model  which  calculates  the  battle  outcomes,  and  a set 
of  sophisticated  interactive  graphics  programs  and  display 
devices  which  allow  two-way  communication  between  the  con- 
trollers and  the  simulation  model. 

Thus  we  have  a closed  loop  system  in  which  the  simulation 
model  calculates  battle  outcomes  and  displays  those  outcomes 
to  the  controllers  as  alphanumeric  messages  on  the  alpha- 
numeric display  devices,  and  also  as  fully  colored  military 
graphic  symbology  overlaying  a full  color  military  map  on 
color  television  monitors.  The  appropriate  role-playing  con- 
trollers relay  information  to  the  trainees  over  the  communi- 
cations system.  The  trainees  react  and  relay  their  orders 
and  requests  for  support  back  to  the  appropriate  controllers 
over  the  communications  system.  The  controllers  use  graphic 
tablets  and  the  color  displays  plus  a complex  set  of  command 
and  control  computer  programs  to  enter  the  full  spectrum  of 
necessary  military  commands  to  the  math  model,  which  updates 
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the  necessary  model  variables  to  carry  out  the  commands,  thus 
changing  all  future  battle  outcome  calculations.  This  closed 
loop,  interactive  system  frees  the  controllers  to  dedicate 
their  efforts  to  role  playing  and  to  the  training  process, 
rather  than  to  the  calculation  of  casualties,  movement  rates, 
etc. ' 

The  basic  CATTS  system  also  includes  an  umpire  or  observer 
monitor  area,  which  allows  students,  senior  officers,  or  ob- 
servers to  monitor  all  aural  communications  and  all  color 
graphics  displays  without  actually  participating  in  the  exer- 
cise. In  this  way  a group  of  students,  for  example,  can 
watch,  listen  and  learn  from  the  mistakes  of  the  trainees 
in  the  TOC. 

A further  system  feature  is  the  simulation  control  sys- 
tem which  allows  the  simulation  to  be  frozen,  replayed,  re- 
started, or  reinitialized  at  the  command  of  the  controllers. 
The  simulation  might  be  frozen  during  classroom  break  periods 
or  for  an  admonitory  warning  over  the  public  address  system. 

A replay  might  be  used  to  show  trainees  (gathered  in  the  um- 
pire area)  what  had  "really"  occurred  and/or  where  they  went 
wrong.  A restart  might  then  be  used  to  reset  the  exercise  to 
a point  just  before  the  "fatal"  error  was  made,  or  to  illus- 
trate the  outcome  using  a preferred  tactic  or  technique.  Re- 
initialization to  the  same  or  a different  scenario  might  be 
used  to  show  that  a tactic  favorable  under  one  set  of  circum- 
stances can  be  disastrous  under  another. 
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Many  different  kinds  of  diagnostic  aids  are  available  to 
assist  the  evaluation  of  trainee  performance.  Status  reports 
are  available  throughout  the  game.  Post-game  processors  pro- 
duce a report  showing  the  change  in  levels  for  every  unit  in 
the  exercise,  also  a summary  of  red  and  blue  casualties,  and 
a sorted  summary  of  all  alphanumeric  messages  produced  during 
the  game.  In  addition,  the  replay  of  selected  portions  of  the 
simulation  with  the  appropriate  color  displays  can  be  a valu- 
able aid  in  reconstructing  the  events  of  the  exercise.  The 
audio  communications  can  also  be  played  back  with  or  without 
the  color  graphics  to  aid  in  evaluating  trainee  communications 
techniques  or  as  an  aid  in  the  reconstruction  of  events. 

B.  THE  SYSTEM  HARDWARE 

The  CATTS  system  is  housed  in  four  separate  rooms — the 
player  room,  the  camera  room,  the  control  room,  and  the  com- 
puter room  (see  Figure  1) . The  only  software  interface  to  the 
player  room  is  with  the  teletype,  which  serves  as  a simulated 
radio-teletype  (RATT) . The  camera  room  contains  three  color 
television  cameras  with  software  controlled  pan,  tilt,  and 
zoom  motors.  The  control  room  contains  the  audio  recorder, 
the  game  clock,  and  three  controller  consoles.  These  con- 
soles are  the  primary  interactive  software  interface.  The 
three  consoles  are  used  as:  a command  and  control  station 
which  oversees  the  entire  conflict,  an  enemy  station,  and  a 
fire  support  station.  The  computer  room  houses  the  computer, 
a XEROX  Sigma  9 Model  Three  and  the  color  graphics  display  de- 
vice. Numerous  other  peripheral  devices  which  serve  to  assist 
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the  controllers  in  the  course  of  the  simulation  are  listed 
in  the  literature  [13]  . 

C.  THE  SYSTEM  SOFTWARE 

The  CATTS  software  is  divided  into  two  types — small, 
fast,  interactive  programs  which  must  have  fast  response  to 
controller  inputs,  and  the  large  CATTS  mathematical  model. 

The  interactive  programs  run  in  the  foreground  mode  [13] , and 
are  principally  concerned  with  communicating  data  and  com- 
mands between  controllers  and  the  simulation  model.  The  si- 
mulation model  (mathematical  model)  runs  in  the  background 
mode,  calculating  battle  outcomes,  casualties,  etc. 

The  CATTS  simulation  model  is  a large,  detailed,  complex 
digital  Monte-Carlo  simulation  of  the  tactical  battlefield 
environment.  It  is  time-step  simulation  model  with  time-steps 
of  one  minute.  It  calculates,  for  each  minute  of  battle,  the 
detections,  engagements,  fires,  casualties,  movement,  and 
environmental  effects  for  up  to  ninety-nine  units.  The  base- 
line scenarios  have  units  that  vary  in  strength  from  a squad 
to  a ballation,  with  the  normal  level  of  platoon  for  friendly 
units  and  company  for  aggressor  units. 

Because  the  model  is  not  interrupt  driven  [13] , it  runs 
as  a background  program,  and  is  often  referred  to  as  the 
"background  software."  The  model  is  functionally  divided 
into  ten  modules,  each  with  a specific  function.  The  fol- 
lowing is  a list  of  these  ten  modules  and  a brief  description 
of  their  implementation: 
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Executive  and  Simulation  Control  Module 
Environmental  Module 
Target  Acquisition  Module 
Ground  Fire  Module 
Ground  Movement  Module 
Engagements  Module 
Input/Output  Module 
Air  Module 

Command  and  Control  Module 
Miscellaneous  and  Ancillary  Module 

1.  Executive  and  Simulation  Control  Module 

This  module  has  the  task  of  overseeing  mathematical 
model  execution.  It  moves  the  correct  overlay  segments  to/ 
from  core  memory  as  required,  directs  the  execution  of  the 
various  other  modules,  handles  the  interface  with  the  fore- 
ground programs,  saves  the  data  necessary  for  replay  and  re- 
start on  disk  files,  and  performs  most  of  the  functions  of  simu- 
lation control. 

2 . Environmental  Module 

The  Environmental  module  has  two  purposes.  One  is 
to  calculate  the  existence  of  lines  of  sight  between  eligible 
ground  units,  considering  terrain  relief  and  vegetation  in- 
teraction. This  is  accomplished  by  a complex  model  using  a 
large  terrain  data  base  developed  from  Defense  Mapping  Agency 
provided  data. 

The  second  purpose  of  the  model  is  to  update  the 
global  weather  conditions,  which  include: 


•temperature 
•relative  humidity 
•weather  class 
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•meteorological  visibility 
•ambient  light  level 
•wind  velocity 
•wind  direction 

3 . Target  Acquisition  Module 

The  target  acquisition  module  determines  the  occur- 
rence of  detections  between  eligible  pairs  of  units  and  gen- 
erates alphanumeric  alert  messages  when  detections  occur. 

Many  environmental  and  tactical  considerations  and  a wide 
range  of  sensor  types  have  been  modeled. 

4 . Ground  Fire  Module 

The  ground  fire  module  is  a complex,  detailed  model 
which  allocates  and  controls  the  fire  of  all  ground  weapons 
modeled  in  CATTS.  It  computes  firing  rates,  casualties,  and 
ammunication  expenditures  for  each  weapon  in  each  unit  each 
time-step. 

5.  Ground  Movement  Module 

The  ground  movement  module  controls  and  directs  the 
movement  of  all  ground  units  in  the  area  of  operations.  At 
each  time-step,  each  unit  is  examined  to  determine  whether  it 
should  start  or  stop  moving.  For  moving  units  a movement 
rate  is  calculated  based  on  tactical  considerations,  existing 
and  new  engagements,  suppression,  and  environmental  factors. 
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Units  may  move  singly  or  as  part  of  an  operational  grouping. 
The  disruptive  effects  of  obstacles  to  movement  rate  are  also 
modeled . 
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6 . Engagements  Module 

The  purpose  of  the  engagements  module  is  to  cause 
ground  units  in  the  model  to  respond  in  a tactically  real- 
istic way  to  enemy  fire  and/or  proximity.  It  determines  when 
units  will  fire  direct  fire  weapons,  when  they  will  form  en- 
gagements, and  when  they  will  break,  them  off. 

7.  Input/Output  Module 

The  input/output  module  consists  of  the  portions  of 
the  model  which  are  concerned  with  input  or  output.  It  in- 
cludes those  routines  which  initialize  the  data  base,  those 
which  produce  the  line  printer  status  reports,  those  which 
generate  alert  messages,  and  those  which  produce  alerts  on 
the  visual  display  monitors. 

8 . Air  Module 

The  air  module  updates  location,  direction,  speed,  and 
altitude  of  each  air  unit  according  to  individually  input 
flight  plans.  This  occurs  at  intervals  of  one-quarter  minute 
or  less.  For  each  quarter-minute  air/ground  interactions  are 
calculated,  including  detections,  firing,  air  weapons  deliv- 
ery, and  casualty  assessment. 

9.  Command  and  Control  Module 

The  command  and  control  module  performs  the  neces- 
sary data  base  updates  for  both  the  interactive  and  the 
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table-driven  command  and  control  in  the  simulation  model. 

The  interactive  portion  processes  the  command  and  control 
event  notices  received  from  the  foreground  Command  and  Con- 
trol Program.  The  table-driven  portion  uses  a tabular  set 
of  input  decision  rules  which  determine  changes  in  unit 
status  if  the  conditions  specified  in  the  table  are  met. 

10.  Miscellaneous  and  Ancillary  Module 

The  miscellaneous  and  ancillary  module  is  a set  of 
service  routines  used  by  other  modules  to  perform  common  cal- 
culations of  such  things  as  line  intersections. 

D.  THE  SIMULATION  MODEL  AND  ITS  DATA  BASE 

From  the  preceding  discussion,  it  is  clear  that  the  heart 
of  the  CATTS  system  is  the  software  and  the  data  base.  The 
modeling  of  a battalion-level  operation  means  that  everything 
of  significance  which  occurs  in  battle  must  be  represented 
in  the  simulation,  i.e.,  a column  of  tanks  heading  toward  an 
objective  must  traverse  various  kinds  of  surfaces  such  as 
roads,  sand,  dirt,  and  swamp.  Each  different  soil  and  road 
type  will  result  in  an  effect  on  the  speed  of  the  column. 

If  a commander  orders  a tank  platoon  through  impenetrable 
vegetation,  it  will  not  make  it  if  the  data  base  and  simula- 
tion model  are  properly  designed.  If  a mortar  platoon  is 
engaging  the  enemy  and  begins  firing  out  of  range,  the  enemy 
attrition  will  be  minimal  or  not  at  all  and,  in  addition,  the 
enemy  will  then  be  aware  of  the  presence  and  location  of  the 
attackers. 
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Due  to  the  si2e  and  complexity  of  the  CATTS  model,  a 
"data-driven"  concept  has  been  employed.  Simply  put,  this 
concept  allows  one  model  (the  model  of  a gun,  for  example) 
to  be  used  for  many  types  of  guns  by  using  different  data 
to  drive  the  model,  resulting  in  a variation  of  weapon  ranges, 
lethality,  firing  rates,  and  aural  levels.  The  attrition  of 
friendly  and  enemy  forces  is  calculated  based  on  many  vari- 
ables including  concentration  of  force,  deployment  of  forces, 
and  weapon  types.  Table  I shows  the  simulation  model  used  in 
CATTS.  Furthermore,  Table  II  and  Table  III  represent  some 
of  the  command  and  control  capabilities  of  the  CATTS  system. 

Real  battles  are  fought  on  real  terrain  and  a knowledge  of 
the  terrain  (soil  type,  relief,  vegetation,  roads  and  rivers) 
is  essential  to  good  tactics.  In  CATTS  the  terrain  data  are 
modeled  mathematically;  events  which  would  normally  be  af- 
fected by  terrain  in  a real  conflict  are  also  affected  in 
CATTS.  Lines-of-sight  from  one  point  to  another  can  be  ob- 
scured by  relief  and  vegetation;  the  movement  of  personnel 
and  equipment  is  a function  of  the  terrain  on  which  they  tra- 
vel. The  commander  who  does  not  know  and  make  good  use  of 
terrain  can  suffer  severely  at  the  hands  of  an  opposing  com- 
mander who  does.  CATTS  presently  has  a digitized  terrain  data 
base  size  of  27  x 100  kilometers.  Elevation  points  are  stored 
every  twenty-five  meters  for  a total  of  4.32  million  eleva- 
tion points.  Elevations  lying  between  the  twenty-five  meter 
coordinate  points  are  calculated  by  interpolation. 
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TABLE  I 

The  computer  simulation  model  contains: 

* Digitized  terrain  data 
(topography,  vegetation,  roads) 

* Weapon  models 
(range,  lethality) 

* Sensors 

(range,  accuracy) 

* Movement  models 
(men  and  equipment) 

* Weather  effects 

* Engagement  of  red  and  blue  forces 

* Line-of-sight  algorithms 
(interact  with  terrain) 

Attrition  algorithms 
(red  and  blue) 

* Automatic  status  report  generation 
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TABLE  II 

Command  and  Control  capabilities  provided  by  the  CATTS  system: 

* Change  the  global  weather  class 

* Execute  a preplanned  mission 

* Deactivate  red  or  blue  units 

* Resupply  red  or  blue  units 

* Perform  red  or  blue  task  organization 

* Create  red  or  blue  air  strikes  or 
air  reconnaissance  missions 

* Issue  red  or  blue  air  defense  commands 

* Maneuver  red  or  blue  units  or  task  forces 

* Create,  move,  delete  red  or  blue  control 
points,  lines,  or  areas 

* Relocate  red  or  blue  units  instantly 

* Issue  red  or  blue  fire/no  fire  commands 
for  any  or  all  weapons  in  a unit 
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TABLE  III 


Simulation  control  capabilities  provided  by  the 
CATTS  System: 

* Reinitialize  the  game  to  any  pre-defined 
scenario 

* Back  up  to  an  earlier  point  in  the  current 
game  and  restart  from  there 

* Back  up  to  an  earlier  point  in  the  present 
game  and  replay  it  exactly  as  it  happened. 
Allow  full  graphics  interactive  capability. 
Replay  at  controller-specified  speed. 

* Terminate  the  replay  in  progress 

* Terminate  the  present  game. 

Print  prespecified  post-game  summaries. 

* Freeze  the  present  game.  Allow  full  inter- 
active graphics  and  command  and  control 
capabilities  during  the  freeze,  but  do  not 
allow  model  to  execute. 


E . CATTS 


THE  TRAINER 
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The  CATTS  system  was  originally  installed  at  Fort  Benning, 

Georgia,  in  March,  1975.  After  successful  acceptance  test- 
ing, the  system  was  used  to  train  severalgroups  of  players 
from  various  areas  of  the  Army.  A player  group  consists  of  up 
to  twenty  personnel,  including  the  battalion  commander,  i.e., 
the  normal  staff  complement  of  a battalion  TOC.  Each  exer- 
cise period  is  presently  four  hours  long  and  the  entire  exer- 
cise, including  tape  recordings  of  player-controller  and 
player-player  conversations  is  preserved  on  mass  storage. 

Replay  of  the  exercise  is  one  of  the  most  significant  attri- 
butes of  CATTS.  Umpires,  who  have  monitored  the  exercise  on 
Cathode  ray  tube  (CRT)  displays  and  then  have  taken  notes  re- 
garding tactical  decisions  made  by  the  player-commander,  at- 
tend the  replay  and  detailed  discussions  of  these  decisions 
are  carried  out  during  the  replay.  The  replay  may  be  backed 
up  or  stopped  at  any  desired  game  time  to  view  specific  areas 
of  the  exercise.  Different  tactical  situations  may  be  used 
to  train  the  players  on  specific  tactical  problems.  Since 
the  CATTS  engagement  game  is  free  play  (units  can  be  anywhere 
and  move  in  any  direction)  various  defense  and  attack  pos- 
tures can  be  structured  to  fine-tune  the  training  as  desired. 

When  one  views  an  exercise  in  progress  from  the  TOC  area  and 
hears  the  static,  the  jamming  and  the  voices  of  the  control- 
lers  acting  as  company  commanders  who  are  receiving  enemy 
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fire,  the  realism  is  quite  convincing.  Since  the  exercise 
is  real  time,  the  decisions  and  actions  of  the  players  must 
also  be  at  the  real-time  rate;  no  time  can  be  wasted  in  lengthy 
analysis  of  fast-breaking  situations  and  efficient  use  of  per- 
sonnel and  time  is  an  important  ingredient  of  the  training 
reg imen. 

The  Army  at  the  present  time  is  evaluating  CATTS  for  pos- 
sible future  uses.  Some  important  questions  associated  with 
possible  future  uses  of  the  CATTS  system  are: 

* How  is  it  best  used  to  train? 

* How  does  it  fit  in  the  whole  training  and  education 
spectrum  of  the  Army? 

* How  can  it  be  used  for  training  troops  on  real  ter- 
rain data  in  a potential  conflict  area  of  the  world? 

* How  can  the  system  be  organized  to  perform  unit 
training  at  remote  posts? 

These  and  other  questions  regarding  the  data  base  and 
some  particulars  of  the  hardware  must  be  answered  before 
CATTS  can  realize  its  full  potential  in  the  training  inven- 
tory of  the  Army.  Planning  is  another  area  in  which  a CATTS- 
like  concept  could  be  of  significant  help.  In  the  same  way 
that  "contingency  training"  would  incorporate  the  particular 
reality  of  the  potential  conflict  area  and  enemy  force  struc- 
ture, the  planning  use  of  CATTS  would  allow  the  decision 
makers  at  division  level,  for  example,  to  plan  the  strategies, 
tactics,  and  contingencies  based  on  data  which  represents  the 
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real  environments  and  force  structures  involved.  Various 
scenarios  could  be  used  which  cover  a reasonable  spectrum 
of  conflict  types  and  intensities,  and  many  of  the  critical 
variables  of  warfare,  such  as  resource  allocation,  lines  of 
communications,  the  use  of  terrain,  enemy  tactics,  etc., 
could  be  analyzed  in  detail. 
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IV.  CONCEPTS  AND  METHODOLOGY  FOR  EVALUATION 
OF  THE  CATTS  SYSTEM 

On  the  surface,  the  Combined  Arms  Tactical  Training 
Simulator  appears  to  be  just  the  educational  and  training 
tool  that  the  Army  has  for  so  long  needed  to  bridge  the 
gap  between  the  seasoned  combat  veterans  and  the  young  in- 
experienced and  untried  officers.  Certainly,  it  would  be 
foolhardy  to  blindly  accept  as  true  all  of  the  claims  made 
for  the  CATTS  system  by  its  developers  and  proponents.  There- 
fore, before  full  scale  implementation  and  utilization  of  the 
system  began  the  Army  set  about  validating  the  CATTS  trainer, 
i.e.,  evaluating  the  entire  system  to  determine  if  it  was 
meeting  the  initial  system  objective  of  providing  a true-to- 
life  simulation  of  the  battalion  commander's  combat  dilemma. 
This  validation  study  began  in  1975. 

While  this  seems  like  a wise  precaution  to  take  before 
a full-scale  implementation  of  the  system  takes  place,  there 
is  a need  for  a much  more  comprehensive  evaluation  of  the 
CATTS  system  and,  specifically,  its  impact  on  training.  It 
would  certainly  be  ingenuous  to  believe  that  the  CATTS  sys- 
tem, once  validated,  was  fully  preparing  the  modern  comman- 
ders for  a future  conflict.  If  the  CATTS  system  was  blindly 
accepted  as  the  ultimate  trainer  and  no  follow-up  studies 
conducted,  then  a fault  or  deficiency  of  the  system  would 
not  make  itself  evident  until  the  next  major  armed  conflict 
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and  that,  as  agreed  unanimously,  is  far  too  late.  There  is, 
undoubtedly,  a need  for  a constant  and  comprehensive  program 
of  evaluation  of  this  system  if  the  Army  is  to  make  full  use 
of  the  CATTS  system  potential.  This  evaluation  is  the  topic 
of  this  chapter. 

The  CATTS  system,  which  was  first  set  up  at  the  CJ.  S. 

Army  Infantry  School,  Fort  Benning,  Georgia,  in  June  1975, 
cost  slightly  more  than  $4  million  to  develop  and  costs  about 
$100,000  a year  to  run,  mostly  in  computer  software  maintenance 
[16] . Some  of  the  original  development  cost  of  $2.7  million 
(under  contract  to  the  Project  Manager  for  Training  Aids  in 
the  Materiel  Development  and  Readiness  Command)  was  in  eighteen 
months'  worth  of  rental  time  on  the  computer  which  the  Army 
later  decided  to  buy  outright  for  about  $600,000. 

The  balance  of  the  original  cost  was  a million  dollars' 
worth  of  software  models  developed  by  TRW  Systems  Group  under 
contract  to  the  Defense  Department's  Advanced  Research  Projects 
Agency.  Some  observers  believe  that  the  system  was  cheap  at 
the  price,  noting  that  it  cost  the  Army  about  $2  million  to 
develop  a new  helmet  (16] . Formal  cost  and  cost  effective- 
ness analyses  are  still  underway,  although  some  CATTS  advo- 
cates fret  that  the  rigid  rules  of  this  process  will  not  ac- 
count for  the  value  of  some  of  the  totally  new  capabilities 
that  the  system  possesses. 

The  purpose  of  this  chapter  is  to  address  the  topic  of 
system  evaluation  for  the  CATTS  system  and,  more  specifically, 
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to  answer  the  following  questions: 

* Why  do  we  want  to  evaluate  the  system? 

* How  should  we  evaluate  it? 

Further,  a test  concept  is  presented  for  the  evaluation  of 
the  CATTS  system. 

A.  WHY 

At  no  other  time  in  history  has  Congress  placed  so  much 
pressure  on  the  armed  services  to  support  and  justify  their 
major  expenditures.  The  clamor  of  public  opinion  and  many 
complex  budgetary  constraints  have  forced  the  Congress  to 
insist  that  all  major  agencies  of  the  Federal  Government  en- 
deavor to  trim  expenditures  to  the  bare  essentials.  This  Con- 
gressional pressure  has  as  one  of  its  more  visible  focal  points 
the  expenditures  of  the  various  armed  services  on  new  and  ex- 
pensive weapons  and  training  systems.  In  order  to  counter 
this  Congressional  pressure,  all  of  the  armed  services  have 
had  to  establish  as  fact  that  all  new  proposed  weapons  and 
training  systems  fill  a definite  and  crucial  need  in  this 
nation's  overall  security  strategy  and  also  that  the  systems 
in  question  meet  or  exceed  their  design  criteria  so  as  to 
justify  the  expenditures  for  their  acquisition. 

The  critical  need,  as  discussed  in  Chapter  Two,  that  the 


CATTS  system  fulfills  is  that  of  training  today's  inexper- 
ienced and  untried  commanders  so  that  they  may  be  able  to 
cope  with  the  complexities  of  future  conflict. 
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B.  HOW 

In  the  discussion  of  the  methodology  for  the  evaluation 
of  a system  such  as  the  CATTS  system,  three  topics  will  be 
dealt  with: 

1.  Measures  of  Effectiveness 

2.  System  Effectiveness 

3.  Training  Effectiveness 
1 . Measures  of  Effectiveness 

Before  a system  such  as  CATTS  can  be  evaluated,  it  is 
essential  that  the  goals  of  that  system  be  clearly  stated  so 
that  the  analyst  possesses  an  adequate  basis  for  comparison 
and  evaluation.  The  primary  goal  of  the  Combined  Arms  Tacti- 
cal Training  Simulator  is  to  improve  the  outcome  of  any  fu- 
ture armed  conflict  through  the  enhanced  preparedness  of  the 
battalion  commander  and  his  staff.  Specifically,  this  pre- 
paredness consists  of  an  increased  proficiency  in  commanding 
and  controlling  a combat  unit  to  its  fullest  potential. 

The  Operations  Research  literature  [18]  is  quite 
specific  as  to  how  to  proceed  with  the  evaluation:  the  analyst 
must  first  select  appropriate  qualitative  measures  of  a sys- 
tem's effectiveness  and  then  the  analyst  must  seek  methods  of 
quantifying  these  measures  so  that  a comparative  evaluation 
of  a system's  worth  can  be  conducted.  This,  unfortunately, 
is  the  most  complex  and  difficult  part  of  the  evaluation 
process.  It  is  a relatively  simple  matter  to  come  up  with 
qualitative  measures  for  a system  such  as  CATTS.  For  example, 
some  qualitative  measures  may  be:  increased  enemy  casualties, 
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decreased  friendly  casualties,  greater  terrain  gains,  im- 
proved battle  outcomes,  etc.  However,  in  order  to  quantify 
these  measures  the  analyst  must  have  access  to  actual  battle- 
field outcomes,  a commodity  that  is  quite  impossible  to  ac- 
quire during  peacetime. 

It  is  for  this  reason  that  "surrogate"  measures  must 
be  used.  In  some  cases,  even  though  a final  valid  evaluation 
cannot  be  made  with  surrogate  measures,  at  least  a preliminary 
tentative  evaluation  can  be  attempted,  until  it  is  possible  to 
acquire  enough  data  to  quantify  the  system  measures  of  effec- 
tiveness. There  are  two  classes  of  surrogates  available: 

a.  Intermediate  (Enabling)  Tasks 

b.  Terminal  Tasks  (Terminal  Evaluation) . 

a.  Intermediate  (Enabling)  Tasks 

Behavioral  scientists  [1,  11,  15,  19]  have  de- 
scribed intermediate  or  enabling  tasks  as  those  tasks  that  must 
be  satisfactorily  accomplished  prior  to  the  attainment  of  a 
major  objective.  For  example,  there  are  several  steps  on  the 
road  to  becoming  an  airline  pilot.  First,  a pilot  must  attain 
a commercial  pilot  certificate,  then  he  must  attain  an  instru- 
ment pilot  rating,  then  a multi-engine  rating,  and  finally  he 
must  attain  an  airline  transport  pilot  certificate.  These  are 
some  of  the  intermediate  tasks  or  requirements  that  must  be 
accomplished  prior  to  becoming  an  airline  pilot.  Similarly, 
there  are  many  varied  intermediate  tasks  that  must  be  mastered 
by  a battalion  commander  and  his  staff  prior  to  successful 
command  of  a combat  unit  is  achieved.  Just  some  of  these 
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tasks  are: 

* Be  thoroughly  knowledgeable  about  capabilities,  effects, 
limitations  and  requirements  of  both  friendly  and  enemy 
weapons  systems . 

* Be  thoroughly  familiar  with  mission,  capabilities, 
tactics  and  probable  courses  of  action  of  enemy  forces. 

* Be  capable  of  effectively  maneuvering  friendly  forces 
in  such  a manner  as  to  enhance  the  probability  of 
victory. 

* Be  capable  of  managing  the  myriad  details  involved  in 
the  coordination  of  fire  support  from  artillery,  heli- 
copter gunships,  tactical  aircraft,  and  naval  gunfire. 

* Be  able  to  accurately  assess  the  situation  on  the  bat- 
tlefield from  the  reports  of  subordinate  and  adjacent 
units. 

These  are  just  a few  of  the  many  intermediate  tasks  that  can 
be  listed  as  being  essential  for  assuring  the  success  of  a 
battalion  commander  on  the  battlefield. 

These  behavioral  scientists  referred  to  previously 
argue  that  armed  with  the  results  of  the  evaluations  of  sig- 
nificant intermediate  tasks  they  can  evaluate  the  relative 
worth  of  a training  system.  This  author  must  agree  that  an 
analyst  can  make  a preliminary  assessment  of  a system's  worth; 
however,  a more  thorough  analysis  must  be  ultimately  based 
on  the  terminal  tasks. 

b.  Terminal  Tasks  (Terminal  Evaluation) 

Terminal  tasks  are  defined  as  those  goals  and 
objectives  that  are  the  designed  intentions  of  a training 
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system.  For  example,  a terminal  task  or  terminal  evaluation 
for  a flight  school  could  be  the  percentage  of  students  who 
successfully  graduate  with  their  pilot  certificates.  With 
regard  to  the  CATTS  system,  the  terminal  evaluation  is  the 
improvement  of  battle  outcomes.  Again,  if  the  medium  for  the 
evaluation  is  actual  combat  itself,  then  results  will  be  im- 
possible to  attain  during  periods  of  peace.  However,  through 
the  use  of  training  devices  such  as  tactical  field  exercises 
it  is  possible  to  simulate,  to  some  degree,  actual  battlefield 
scenarios  and  thus,  to  some  degree,  attain  an  evaluation  of  the 
battalion  commander's  ability  to  command  and  control  his  unit. 

It  is  the  opinion  of this  author  that  since  the  overall 
system  objective  of  the  CATTS  trainer  is  to  improve  the  out- 
come of  battles,  it  would  be  more  appropriate  to  study  the 
results  of  a terminal  task  evaluation.  Certainly,  the  CATTS 
trainer  would  conceivably  score  well  if  evaluated  on  the  ba- 
sis of  the  intermediate  or  enabling  tasks,  but  this  is  not  the 
goal  of  the  CATTS  system.  Instead,  the  Combined  Arms  Tacti- 
cal Training  Simulator  is  designed  to  take  the  battlefield 
commander's  skill  in  accomplishing  these  intermediate  tasks, 
exercise  and  improve  these  skills,  and  teach  the  commander 
how  to  use  these  skills  simulatneously  and  integrate  them 
through  the  experiential-type  training  situation.  The  objec- 
tive of  the  CATTS  system  is  to  improve  the  "fighting  ability" 
of  units:  it  is  precisely  this  factor  that  analysts  should 
test  for  in  the  evaluation  results. 
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J.  G.  Taylor  has  written  [18] : 


"The  effectiveness  of  any  military  system  may 
be  defined  as  the  extent  of  success  to  which  the 
system  may  be  expected  to  achieve  a set  of  objec- 
tives. These  objectives  may  either  be  explicitly 
stated  as  specific  mission  requirements  or  be  im- 
plicit in  the  system's  operational  deployment. 

The  objectives  of  the  system  are  frequently  deter- 
mined by  its  relations  with  higher  echelon  systems. 
Judgment  and  experience  must  be  exercised  by  the 
analyst  to  determine  the  extent  to  which  these 
’super  systems'  (and  also  subsystems)  must  be  con- 
sidered for  identifying  objectives.  The  defining 
of  the  system  and  identification  of  its  objectives 
is  the  development  of  a qualitative  effectiveness 
concept. 

"The  quantitative  expression  of  the  extent  to 
which  specific  mission  requirements  are  attained  by 
the  system  is  referred  to  as  'a  measure  of  effective- 
ness.' The  choosing  of  measures  of  effectiveness 
for  a system  under  consideration  is  most  success- 
fully done  by  quantifying  a previously  conceived 
qualitative  effectiveness  concept. 

"Very  often  failure  to  choose  the  appropriate 
measures  of  effectiveness  can  lead  to  completely  wrong 
conclusions  as  to  preferred  alternatives.  This  usu- 
ally has  resulted  not  from  failure  to  choose  measures 
which  describe  system  performance  but  from  lack  of 
adequate  consideration  of  system  objectives.  Fre- 
quently, the  most  desired  objectives  of  system  per- 
formance are  not  explicitly  stated.  It  should  also 
be  noted  that  the  objectives  of  a system  may  vary 
with  its  mode  of  operation  or  type  of  mission." 

Thus,  the  ultimate  objective  of  the  CATTS  system 
should  be  constantly  referenced  during  the  selection  of  ap- 
propriate measures  of  effectiveness  for  its  evaluation.  This 
ultimate  objective  is  the  improvement  of  battle  outcomes,  and 
it  is  for  the  same  reasons  that  Taylor  has  given  that  this 
author  supports  the  evaluation  of  the  CATTS  system  based  on 
the  terminal  task  evaluation. 

2.  System  Effectiveness 

P.  Hayward  [18]  has  outlined  four  factors  upon  which 

the  system  effectiveness  of  a military  training  system 
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depends:  a.  Enemy  Force/Hardware  Capabilities 

b.  Friendly  Force/Hardware  Capabilities 

c.  The  Combat  Environment 

d.  The  Mission. 

a.  Enemy  Force/Hardware  Capabilities 
The  training  system  must  be  capable  of  accurately 

representing  modern  enemy  tactics  and  weapons  employment/ef- 
fects in  order  to  give  the  trainees  a realistic  picture  of  what 
it  is  they  must  defeat.  All  relevant  enemy  capabilities  must 
be  portrayed  and  integrated  into  the  scenario  in  order  that 
the  training  be  truly  adequate.  Further,  the  training  system 
must  be  capable  of  being  updated  so  that  future  changes  in 
enemy  tactics,  weapons,  or  capabilities  can  also  be  repre- 
sented. 

b.  Friendly  Force/Hardware  Capabilities 
The  training  system  must  be  of  sufficient  detail 

so  that  the  trainees  are  acquainted  with  all  of  the  force 
capabilities  and  the  hardware  capabilities  available  to  them 
during  an  armed  conflict.  Further,  the  training  system  must 
be  capable  of  accurately  portraying  the  effects  of  both  ef- 
fective and  ineffective  use  of  these  capabilities,  as  well 
as  the  synergistic  effects  derived  from  the  use  of  comple- 
mentary tactics  and  complementary  hardware  systems. 

c.  The  Combat  Environment 
The  training  system  must  be  capable  of  depicting 

all  of  the  effects  that  environmental  elements  have  upon  the 
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outcome  of  battles.  For  example,  it  must  be  capable  of  ac- 
curately portraying  the  terrain,  vegetation,  relief,  weather, 
weapons  effects,  etc.,  and  their  impact  upon  the  final  out- 
come. Additionally,  the  training  system  must  be  capable  of 
differentiating  between  a trainee  that  makes  wise  use  of 
these  environmental  factors  (for  example,  terrain)  and  that 
trainee  who  overlooks  their  importance, 
d.  The  Mission 

For  the  trainee  to  truly  be  prepared  for  future  con- 
flicts, he  must  be  allowed  the  opportunity  to  practice  dealing 
with  those  factors  which  he  is  most  likely  to  encounter  on  the 
battlefield.  Thus,  the  system  must  be  capable  of  simulating 
the  various  combat  mission  types  that  history  indicates  will 
most  likely  confront  the  trainee  battalion  commander — such 
as  attack,  defend,  delay,  screen,  cover,  withdraw,  etc. 

This  is  the  advantage  of  a training  system  such  as 
CATTS;  the  ability  to  expose  a trainee  to  the  same  rigors 
and  situations  that  he  will  be  exposed  to  in  battle.  In  order 
for  a training  system  like  this  to  be  effective,  it  must, 
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as  a minimum,  consider  and  satisfactorily  deal  with  the  four 
factors  mentioned  above,  allowing  the  trainee  to  cope  with 
the  simulated  crises  and  difficult  decisions  at  the  same 
"man-killing"  tempo  that  he  will  encounter  on  the  battlefield. 

3 . Training  Effectiveness 

Training  effectiveness  has  been  defined  as  player 
acquisition  of  new  skills  and  knowledge  and/or  improvement 
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in  performing  existing  skills  and  knowledge  that  result 
from  the  interaction  with  a training  system  [10] . The  train- 
ing effectiveness  of  a system  must  be  evaluated  in  order  to 
insure  that  the  training  system  is  indeed  capable  of  provid- 
ing the  training  desired  and  that  the  capabilities  of  the 
training  system  are  fully  implemented  to  adequately  train 
the  students.  The  goal  here  is  to  have  the  trainee  learn  as 
much  from  the  training  system  as  if  he  had  participated  in 
actual  combat  himself. 

Training  effectiveness  is  considered  a function  of 
four  training  effectiveness  indicators  [10] : 

a.  Player  Exercise  Capability 

b.  Training  Process  Management 

c.  Training  Vehicle  Repertoire 

d.  System  Reliability,  Availability,  and 
Maintainability . 

a.  Player  Exercise  Capability 

Player  exercise  capability  is  the  extent  to  which 
a training  system  permits  players  to  exercise,  practice  and 
apply  skills,  knowledge,  and  techniques  to  be  developed 
through  the  training.  A training  system,  to  achieve  any 
degree  of  effectiveness,  must  permit  the  player  personnel 
an  opportunity  to  practice  or  apply  skills  or  techniques  to 
be  acquired.  This  facilitates  the  acquisition  process  by 
providing  player  performance  samples  for  self  and  instructor 
critique  and  the  opportunity  to  take  corrective  action. 

Some  other  considerations  are: 


(1)  Problem  Perception 

Player  problems  are  generated  by  training 
objectives.  A player  is  required  by  the  training  exercise  to 
recognize  the  existence  of  a problem  in  his  area  of  responsi- 
bility. He  must  take  action  on  it,  to  include  making  deci- 
sions, and  arrive  at  a solution  following  procedures  appro- 
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priate  to  his  responsibilities,  using  resources  available  to 
him,  and  taking  into  account  the  parameters  of  the  tactical 
situation.  This  process  is  required  to  teach  the  player  the 
skills,  knowledge,  decisions,  and  actions  required  of  his 
duty  position,  and  to  provide  a work  sample  for  evaluation 
of  his  level  of  mastery  of  required  job  elements. 

To  accomplish  a training  or  performance  ob- 
jective and  to  assess  player  performance  adequacy,  players 
must  be  confronted  with  command  and  control  problems  to  be 
solved  during  the  course  of  a scenario.  A player  will  be 
confronted  with  stimuli  indicating  the  presence  of  a problem 
requiring  decision  or  action.  The  player  must  recognize  the 
stimuli,  differentiate  relevant  from  non-relevant  stimuli, 
correctly  interpret  the  information,  and  initiate  actions 
he  deems  appropriate.  The  training  system  must  have  the 
capability  to  present  the  required  stimuli. 

A training  system  should  provide  the  means  for 
a player  to  take  realistic  actions  on  scenario  events.  Such 
actions  may  be  correct,  incorrect,  or  irrelevant  with  respect 
to  events.  The  response  system  should  permit  the  player  to 


5 


take  actions  peculiar  to  his  duty  position  to  demonstrate 
that  he  has  detected  a problem  and  he  is  taking  action  on 
it,  even  though  such  actions  may  be  incorrect.  Feedback  or 
knowledge  of  the  results  is  necessary  to  inform  players  how 
well  they  are  performing  in  order  to  develop  the  neces  sary 
skills.  Such  feedback  should  be  realistic  and  in  a form  of 
minimum  delay?  however,  in  a real  time  scenario  a realistic 
latency  period  is  desirable. 

Acquisition  of  command  and  control  skills 
and  techniques  is  also  a function,  to  a certain  extent,  of  the 
amount  of  time  the  players  participate  in  training  exercises. 
The  number  of  players  to  be  trained  may  restrict  the  amount 
of  time  the  players  can  utilize  a training  system  and  associ- 
ated strategy,  thereby  reducing  the  exercise  capability  and 
training  effectiveness. 

(2)  Player  Preparation  Requirement 

The  player  preparation  requirement,  simply 
stated,  is  that  the  effectiveness  of  the  training  system  may 
depend  on  the  professional  (military)  proficiency,  experience, 
and  formal  training  of  the  participant. 

b.  Training  Process  Management 

To  insure  that  intended  training  objectives  and 
performance  standards  are  attained  by  the  players,  the  train- 
ing process  must  be  managed  or  controlled  by  the  controllers/ 
instructors.  Such  control  entails  controller  monitoring, 
evaluation,  and  critiqueing  of  player  performance  and 
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direction  of  the  training  process  to  insure  training  objec- 
tives are  met  and  performance  deficiencies  are  corrected. 

Additionally,  the  controllers  must  exert  control 
over  the  training  process  to  implement  a training  strategy. 

The  controllers  are  responsible  for  establishing  conditions 
that  elicit  the  performance  required  to  insure  acquisition 
and  mastery  of  required  skills,  techniques,  and  procedures 
called  out  in  performance  objectives. 

Players  learn  command  and  control  skills  and  tech- 
niques by  practice  under  scenario  conditions,  reinforced  by 
feedback.  In  addition  to  the  self-evaluation  permitted  by 
the  scenario,  players  can  be  critiqued  by  the  controllers 
concerning  how  well  they  met  performance  objectives.  In 
order  to  critique  players,  the  controllers  must  have  the 
capability  to  monitor  performance  and  evaluate  it.  Some 
other  considerations  are: 

(1)  Problem  Control 

Problem  control  is  initiated  with  scenario 
development.  At  this  stage,  events  designed  to  implei.:ont 
training  and  performance  objectives  are  built  into  the  sce- 
nario. A scenario  should  provide  for  contingency  actions  in 
case  the  players  deviate  from  the  expected  courses  of  action. 

A set  of  baseline  conditions  are  required  for  a point  of  de- 
parture for  a training  scenario.  These  are  used  to  read  the 
player  into  the  problem  (initial  briefing) . Subsequent  events 
used  to  develop  teaching  points  and  training  objectives  build 
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upon  the  initial  conditions.  A minimum  level  of  clarity  is 
required  so  that  the  player  understands  the  problem.  The 
ease  or  complexity  of  establishing  initial  conditions  will 
vary  with  the  training  system.  Further,  controllers  must 
monitor  the  progress  of  the  training  scenario  to  insure  that 
the  scenario  is  operating  according  to  plan  and  training 
strategy,  and  to  determine  the  necessity  for  implementing 
contingency  plans. 

The  controllers  have  the  responsibility  to 
insure  that  training  objectives  and  performance  objectives 
are  attained.  This  requires  that  they  control  the  direction  of 
the  scenario.  Further,  in  order  to  shorten  training  time  it 
may  be  necessary  to  change  the  scenario  time  base  during  a 
training  session  from  real  time  to  compressed  time  and  then  re- 
establish real  time.  Ease  of  this  accomplishment  with  minimum 
disruption  is  a consideration  in  training  system  utilization 
and  evaluation. 

(2)  Player  Evaluation 

To  assure  training  objectives  and  performance 

objectives  are  met,  controllers  must  monitor  and  evaluate 

player  actions  against  performance  standards.  The  ease,  time- 
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liness,  and  completeness  of  performance  evaluation  are  deter- 
mined by  training  system  characteristics.  Knowledge  of  per- 
formance is  utilized  to  inform  players  how  well  they  are  per- 
forming in  command  and  control  training  scenarios.  This  in- 
formation is  used  to  assist  players  in  improving  performance 
in  areas  of  deficiency  and  to  motivate  players. 
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A communications  system  is  required  to  sup- 
port control  of  the  program  of  instruction  and  the  scenario 
and  to  facilitate  player  evaluation.  The  effectiveness  of  a 
training  vehicle  may  depend  on  the  type  of  communications 
system  used. 


(4)  Controller  Requirement 

Controllers  should  meet  certain  minimum  exper- 
ience and  training  standards.  In  order  to  qualify  as  control- 
lers of  a certain  training  system,  they  should  be  intimately 
familiar  with  that  training  system  and  its  peculiarities, 
c.  Training  Vehicle  Repertoire 

•Given  a set  of  required  training  objectives,  a 
training  vehicle  (i.e.,  command  post  exercise,  map  exercise, 
etc,)  intended  to  mediate  accomplishment  of  those  objectives 
should  have  the  inherent  capability  to  support  the  necessary 
practice  exercises  and  training  strategies  at  some  minimum 
level.  The  training  vehicle  should  not  be  too  complicated 
to  use  and  it  should  be  capable  of  being  updated  or  modified 
to  reflect  changes  in  tactical  doctrine,  concepts,  and  organi- 
zation. Some  further  considerations  are: 

(1)  Data  Base 

Utilization  of  a training  system  and  associ- 
ated program  of  instruction  requires  data  and  documentation 
support.  Completeness  of  these  factors,  ease  of  their  use, 
and  time  required  to  use,  can  affect  training  effectiveness 
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directly  or  ir directly.  It  can  likewise  affect  the  training 
costs.  The  relative  ease  or  difficulty  of  updating  training 
scenarios  should  be  assessed.  This  impacts  on  operational 
economics  and  personnel  skills  required  to  support  a training 
system. 

(2)  Problem  Configuration  Capability 

This  is  the  number  and  type  of  tactical  event 
elements  and  combinations  of  elements  available  for  developing 
stimulus-response  events  for  a command  and  control  tactical 
training  scenario  for  use  with  a training  system.  This  would 
define  the  tactical  combat  model  in  a computerized  system 
(such  as  CATTS) . Scenario-based  instruction  requires  repre- 
sentation of  friendly  forces  and  enemy  forces  in  some  degree 
of  detail  determined  by  training  requirements.  More  signifi- 
cantly, this  type  of  scenario-based  training  requires  repre- 
sentation of  most  types  of  tactical  operations  for  purposes 
of  command  and  control  practice  so  that  players  can  learn  dif- 
ferences in  command  problems  that  are  associated  with  differ- 
ent kinds  of  operations.  Some  functions  that  the  system  will 
have  to  perform  using  scenario-based  instruction  are:  battle 
and  casualty  assessment,  mobility  assessment,  representation  of 
movement  rates,  terrain,  vegetation,  visibility,  weather,  and 
climate. 

(3)  Evaluation  Feedback  Aids 

Performance  evaluation  requires  that  a train- 
ing system  permit  evaluation  of  player  performance  by  the 
controllers  and  the  player  himself.  The  effectiveness  of  a 
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training  system  depends  in  part  on  how  accurately,  completely, 
and  timely  this  evaluation  can  be  made.  This  in  turn  is  fa- 
cilitated by  the  ease  of  which  the  controller  or  player  can 
assess  this  information.  In  some  cases  a training  system  may 
preprocess  or  "highlight"  certain  performances  as  an  aid  to  a 
controller  or  player.  This  capability  of  a system  should  be 
assessed.  Further  points  the  system  should  be  scrutinized  on 
are: 

* The  degree  of  completeness  of  feedback  to  players 
provided  by  certain  systems. 

* It  is  desirable  that  a training  system  have  a recording 
and  replay  capability  for  purposes  of  critiqueing  players. 

* It  is  desirable  that  feedback  to  players  be  in  the  form 
of  battle  outcomes.  This  is  realistic  and  provides  the 
players  with  realistic  information  concerning  the  con- 
sequences of  their  command  decisions. 

(4)  Operational  and  Processing  Time 

It  is  desirable  that  a training  system  operate 
in  real  time  or  apparent  real  time  for  the  sake  of  realism.  F5r 
training  strategy  reasons  and  reasons  of  economy  it  may  also  be 
desirable  that  the  system  operate  in  compressed  time.  This 
capability  should  be  assessed. 

d.  System  Reliability,  Availability,  and  Maintain- 
ability 

A training  system  must  operate  with  little  or  no 
failure  and  delay  to  insure  that  the  training  process  is  not 
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disturbed,  that  training  is  performed  as  scheduled,  and  that 
the  program  of  instruction  is  successfully  completed.  Some 
further  considerations  are: 

(1)  Reliability 

Lack  of  training  vehicle  reliability  can 
lead  to  failure  to  accomplish  training  objectives  and  per- 
formance objectives.  Unreliability  is  also  undesirable  be- 
cause training  is  usually  tightly  scheduled  and  may  impose  an 
undue  maintenance  burden  on  the  system  support  personnel. 

(2)  Availability 

Training  system  non-availability  or  delay 
is  undesirable.  This  factor  can  result  in  training  objectives 
and  performance  objectives  not  being  met,  increases  in  train- 
ing time  and  costs,  and/or  failure  to  complete  training. 

(3)  Maintainability 

A training  system  should  not  impose  an  undue 
burden  on  maintenance  personnel.  A high  burden  such  as  this 
could  render  the  system  an  uneconomical  means  of  training. 

When  considering  the  overall  training  effective- 
ness of  a training  system  these  four  points  above  must  be 
considered  in  conjunction  with  the  training  strategy  employed 
with  the  system.  A training  strategy  is  a set  of  procedures 
employed  to  guide  player  learning  and  to  attain  the  stated 
training  objectives.  Several  alternative  training  strategies 
may  be  employed  with  a training  system  unless  the  character- 
istics of  the  system  limit  the  strategy  alternatives  to  one. 
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The  training  system  performance  can  be  optimized  by  a properly 
designed  and  implemented  strategy  or  it  can  be  degraded  by  im- 
properly conceived  or  implemented  training  strategies.  The 
current  CATTS  training  strategy  calls  for  utilization  of  an 
experiential-type  training  strategy  for  players. 

To  summarize,  the  evaluation  of  a training  system  must 
first  start  out  with  the  clear  statement  of  the  training  system's 
objectives  followed  by  the  statement  of  the  appropriate  quali- 
tative effectiveness  concepts.  An  effort  must  be  made  to  quan- 
tify these  qualitative  measures  into  measures  of  effective- 
ness and  if  this  cannot  be  accomplished,  then  surrogate  mea- 
sures must  be  used.  Finally,  a training  system  is  further 
evaluated  on  its  inherent  system  effectiveness  factors  in  con- 
junction with  the  four  training  effectiveness  indicators. 

Through  the  integrated  evaluation  of  the  above  factors  the 
analyst  will  be  able  to  reach  a satisfactory  evaluation  of  the 
training  system,  one  that  can  be  used  to  compare  the  training 
system  in  question  with  others. 

C.  THE  TEST  CONCEPT 

The  best  system  found  to  date  by  the  Army  for  command  and 
control  training  at  the  battalion  level  is  the  conventional 
command  and  staff  training  system  currently  employed  at  the 
Command  and  General  Staff  College,  Fort  Leavenworth,  Kansas. 

This  type  of  training  uses  the  classroom  techniques  of  lec- 
ture, conference,  demonstration,  and  practical -exercise  methods 
of  instruction,  e.g.,  map  exercises.  The  curriculum  that  is 
to  be  tested  against  the  CATTS  system  can  be  designed  to 
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accomplish  the  same  training  performance  objectives  as  the 
CATTS  training  strategy  and  it  can  be  conducted  within  the 
same  time  constraints.  For  purposes  of  test  completeness, 
the  performance  of  CATTS-trained  personnel  and  conventionally 
trained  personnel  can  be  compared  with  the  performance  of  per- 
sonnel who  have  had  no  other  command  and  control  training  other 
than  that  contained  in  the  appropriate  Advanced  Course.  This 
group  can  be  used  as  a control  group. 

The  comparative  analysis  is  designed  to  answer  the  fol- 
lowing training  performance  issues: 

1.  Do  CATTS  and  alternative  training  systems  improve 
command  and  control  performance? 

2.  Do  CATTS  and  conventional  training  approaches 
differ  in  training  effectiveness? 

3.  Does  prior  training  and  experience  of  the  player 
affect  training  performance? 

4.  Do  alternative  training  systems  differ  in  terms  of 
uniformity  of  the  end-of-training  performance 
produced? 

5.  Do  CATTS  training  and  alternative  training  approaches 
benefit  on-the-job  performance  of  incumbent  staffs? 
Which  training  system  yields  the  greatest  gains? 

This  analysis  can  be  easily  carried  out  using  the  standard 
statistical  techniques  only  after  the  appropriate  measures  of 
effectiveness  have  been  developed.  To  date,  no  such  test  has 
been  performed.  It  is  the  opinion  of  this  author  that  when 
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this  kind  of  analysis  is  finally  completed,  the  CATTS  sys- 
tem will  be  found  to  enhance  the  performance  and  the  overall 
preparedness  of  future  commanders  to  deal  with  future  battles. 
More  importantly,  it  is  also  this  author's  opinion  that  the 
best  overall  training  effectiveness  will  be  attained  through 
a judicious  combination  of  the  two  training  systems:  the 
conventional  system  to  give  the  trainees  a solid  background 
in  the  fundamentals  and  the  CATTS  system  to  allow  the  trainee 
to  put  his  new-found  knowledge  to  use  and  to  sharpen  his  in- 
stincts and  reactions  in  a true-to-life  simulation  of  a battle. 
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V.  SUMMARY  AND  FINAL  COMMENTS  ON  CATTS 

The  Combined  Arms  Tactical  Training  Simulator  indeed 
promises  to  bring  a new  dimension  to  command  and  control 
training  for  the  U.  S.  Army.  Previously,  students  were 
constrained  to  what  they  could  learn  from  classroom  lectures, 
seminars  and  the  appropriate  literature.  However,  with  the 
appearance  of  the  CATTS  system  the  command  and  control  stu- 
dent can  now  step  into  an  environment  which  reproduces  the 
stresses,  sights  and  sounds  of  combat  command  as  closely  and 
realistically  as  ever  before. 

More  specifically,  some  of  the  other  apparent  benefits 
of  the  CATTS  system  are: 

* Eliminates  expensive  operation  and  maintenance  of 
real  equipment. 

* Saves  the  time  and  logistics  costs  of  deploying 
troops  in  field  exercises. 

* Conserves  instructor  manpower  in  preparing  the  con- 
ducting command  and  control  training  courses. 

* Provides  more  practice  under  a greater  variety  of 
conditions  than  is  feasible  in  field  exercises. 

* Furnishes  realistic  training  in  geographical  areas 
where  live  training  is  not  possible. 

* Provides  training  in  new  tactics  and  procedures. 

* Tests  both  the  planning  and  the  execution  of  orders 
in  combat. 
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At  the  time  of  this  writing,  the  CATTS  system  was  in- 
stalled at  the  Command  and  General  Staff  College  at  Fort 
Leavenworth,  Kansas,  where  it  has  been  integrated  into  the 
curriculum  for  the  purpose  of  training  battalion  commanders 
of  the  future.  Work  is  also  underway  to  make  the  CATTS  sys- 
tem available  to  other  military  bases  around  the  country  and 
around  the  world  so  that  incumbent  staffs  could  train  together 
with  their  commanders  and  develop  the  required  skills  and 
techniques  necessary  for  battlefield  success.  Studies  are 
also  underway  to  evaluate  the  possible  use  of  the  CATTS 
trainer  as  a battalion  staff  evaluation  tool,  since  there 
is  a fine  line  between  training  and  evaluation. 

CATTS  permits  future  commanders  and  their  staffs  to 
practice  commanding  and  controlling  their  units — in  real 
time  under  the  most  realistic  simulated  conditions  available. 
The  CATTS  system  is  yet  to  be  thoroughly  proven,  however, 
the  future  bodes  well  for  it. 

To  summarize,  this  thesis  has  attempted  to  acquaint  the 
reader  with  the  Combined  Arms  Tactical  Training  Simulator  it- 
self,  as  well  as  the  rationale  for  its  creation.  A detailed 
study  of  the  CATTS  system  and  its  major  components  was  pre- 
sented. Finally,  the  concepts  and  methodology  appropriate 
to  the  evaluation  of  a system  such  as  CATTS  were  presented. 
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